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GEOGRAPHIC  INFORMATION  SYSTEMS  IN  ROBOTIC  VEHICLES 

by 

Bruce  B.  Zimmerman,  Richard  L.  Roaenthal  and  Jerry  M.  Breen 

U.S.  Army  Engineer  Topographic  Laboratories 
Fort  Belvolr,  Virginia  22060-5546 


ABSTRACT 

Robotic  v^lclea  offer  potential  to  the  Army  for  force  multiplication  and 
removing  the  soldier  from  hazardous  environments.  The  Army  robotic  vehicle 
program  will  provide  a  supervised  autonomy  capability  with  full  battlefield 
mobility.  To  accomplish  this  the  control  system  will  require  a  person  In 
the  control  loop  at  least  part  of  the  time.  The  Army  program  Incorporates 
several  technologies  Including  a  digital  terrain  data  base,  product  generation 
capabilities,  stereo  vision,  and  an  Inertial  position  and  attitude  unit. 
These  technologies  allow  for  vehicle  operation  over-the-hlll  and  beyond 
llne-of-slght.  This  paper  will  discuss  the  detailed  use  of  terrain  data 
and  terrain  data  products,  such  as  route  planning  and  three-dimensional 
synthetic  scenes.  In  the  Interactive  control  of  a  robotic  vehicle.  Of 
particular  importance  la  the  relation  of  the  terrain  data  to  the  vision 
data  and  position  and  attitude  data.  This  paper  will  also  discuss  the 
similarities  and  differences  between  GIS  requirements  for  Interactive 
robotic  vehicle  control  and  autonomous  vehicles,  such  as  lL'n  the  DARPA 
Autonomous  Land  Vehicle  program.  ■  •  • 

\. 

INTRODOCTION 

Army  robotics  programs  officially  began  In  1981  when  the  Office  of  the  Deputy 
Chief  of  Staff  for  Research,  Development  and  Acquisition  (DCSRDA)  In 
Headquarters,  Department  of  the  Army  formed  an  artificial  Intelligence 
(AI)/Robotlcs  Steering  Committee.  This  committee  commissioned  a  survey  of 
the  Army  User  and  Development  communities  on  the  use  of  this  new  technology 
called  Al/Robotlcs  which  resulted  in  a  prioritized  list  of  hundreds  of 
potential  applications.  The  results  of  this  survey  were  briefed  to  the  top 
levels  of  Army  management,  which  endorsed  the  basic  concepts  but  recommended 
focusing  Army  resources  on  a  few  selected  application  areas. 

Following  this  guidance  Initial  plans  were  drawn  up  In  1982  for  several 
near-term  demonstration  systems  with  the  potential  for  future  growth.  One 
of  the  Initial  proposals  was  for  an  unmanned  vehicle  capable  of  carrying 
various  modular  payloads  configured  for  specific  missions.  These  could 
Include  battlefield  reconnaissance,  nuclear,  biological,  chemical  (NBC) 
detection,  weapons  modules,  target  designators,  etc.  There  was  a  desire  to 
develop  a  fully  autonomous  system,  at  least  as  far  as  mobility  Is  concerned, 
to  minimize  manpower  and  communications  requirements. 
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However,  technological  reality  dictated  that  true  autonomy  vaa  decades  away 
so  the  plana  called  for  a  teleoperated  control  system  to  provide  a  near-term, 
military  capability.  While  the  system  would  require  a  person  to  be  In  the, 
control  loop  full  time,  the  primary  goals  of  vehicle  robotics  could  be! 
accomplished:  1)  to  remove  the  soldier  from  potentially  hazardous  environ-; 
ments;  and  2)  to  provide  for  force  multiplication  by  being  able  to  field  aj 
higher  ratio  of  vehicles  to  crew  size  than  Is  possible  today. 


In  addition  the  initial  plans  called  for  incorporation  of  key  technologies 
to  allow  for  over  the  hill,  non  vlsual-llne-of-slght  control  and  for 
evolutionary  growth  toward  autonomy;  l.e.,  to  gradually  reduce  the  amount 
of  time  the  person  Is  required  In  the  control  loop.  Among  these  critical 
technologies  are  the  use  of  digital  terrain  data  bases  and  derived  products 
In  conjunction  with  stereo  vision  and  an  inertial  measurement  unit. 


This  paper  will  describe  both  Interactive  and  autonomous  robotic  vehicle 
control  systems  and  the  role  that  terrain  data  bases  and  products  derived 
from  the  terrain  data  bases  play.  These  products  rely  on  the  use  of 
geographic  Information  systems  (GIS)  and  by  way  of  example  the  use  of  GIS 
In  robotic  vehicles  will  be  Illustrated. 


ROBOTICS  BACKGROUND 


The  basic  plans  for  the  robotic  vehicle  program  have  undergone  significant 
evolution  since  the  formative  years  of  1982  through  1984.  The  program  Is 
now  called  the  Supervlse^d  Autonomy  Testbed  (SAT)  and  Is  under  the  direction 
of  an  Interlaboratory' management  team.  The  Army's  vehicle  developer,  the 
Tank-Automotive  Command  (TACOM)  Is  the  lead  lab  and  the  Engineer  Topographic 
Laboratories  (ETL)  and  the  Human  Engineering  Laboratory  (HEL)  are  the  other 
two  primary  members. 


Figure  1  shows  an  artist's  conception  of  the  vehicle  system  In  operation. 
In  the  background  Is  the  unmanned  platform  equipped  with  remote  actuators, 
an  Inertial  unit,  stereo  cameras,  and  a  reconnaissance  module  to  accomplish 
mission  objectives.  In  the  foreground  Is  the  remotely  located  command  and 
control  station  where  the  person  driving  the  vehicle,  and  perhaps  a  second 
person  monitoring  the  recon  sensor  display,  are  located.  A  stationary  van 
will  be  used  for  demonstration  purposes  but  any  fielded  system  would  place 
the  command  and  control  station  in  a  mobile  platform  also.  The  computers, 
displays  (graphic  and  stereo  vision),  and  digital  terrain  data  base  are 
located  In  the  command  and  control  station.  A  non  visual-llne-of-slght 
communications  system  will  provide  the  link  between  the  unmanned  platform 
and  the  command  and  control  station  to  allow  over  the  hill  operation. 

Figure  2  shows  an  artist's  concept  of  the  driver's  station  In  the  command 
and  control  center.  The  lower  left  display  depicts  a  (digital)  map  of  the 
area  of  operation,  a  preplanned  optimal  route  derived  from  this  a  priori 
knowledge,  and  a  blinking  cursor  overlay  derived  from  the  x,  y,  and  heading 
data  from  the  Inertial  unit.  The  two  upper  displays,  which  are  Intended 
to  convey  the  Idea  of  a  stereo  vision  display,  provide  the  primary  sensory 
Input  for  driving  the  vehicle.  The  blinking  cursor  display  resorts  to  a 
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navigation  aid  after  prenlaalon  planning  haa  been  accoapllahed.  The  final  I 
display,  In  the  lower  right  corner,  is  a  sinpllstic  rendition  of  what  the  [. 
terrain  along  critical  portions  of  the  route  alght  look  like  to  the  driver,  i 

In  thia  person-in-the-control-loop  scenario  the  huaan  la  either  perfonlng  j 
or  supervising  decision  asking  procesaea  such  as  approving  the  preplanned  j 
route,  interpreting  the  stereo  vision,  detecting  obstacles  and  planning  ' 
local  path  aaneuvering  around  the  obstacles,  perfomlng  landaark  recognition, 
aaintaining  aisslon  progress  in  the  context  of  the  preplanned  route,  and 
even  replanning  the  global  route  when  the  situation  warrants.  If  the 
person  makes  all  of  these  decisions  then  the  system  is  pure  teleopera- 
tlon.  If  the  computer  is  able  to  make  some  of  the  decisions  but  able  to 
call  on  the  human  for  help  when  needed,  then  the  system  is  exercising 
supervised  autonomous  control. 


If  a  machine  can  be  made  Intelligent  enough  to  peform  all,  or  substantially 
all  of  the  decision  making,  then  the  system  would  be  termed  autonomous. 
Today's  technology,  and  perhaps  even  tomorrow's,  does  not  provide  the  machine 
intelligence  to  autonomously  maneuver  through  the  complex,  unstructured 
environment  of  battlefield  terrain.  However  the  Defense  Advanced  Research 
Projects  Agency  (DARPA)  is  tackling  this  high  risk,  but  potentially  high 
payoff  problem  with  their  Autonomous  Land  Vehicle  (ALV)  program,  figure  3. 
ETL  is  the  DARPA  agent  for  this  program  which  provides  a  focus  for  the 
comprehensive  research  being  conducted  in  the  Strategic  Computing  Program 
of  which  the  ALV  is  an  Integral  part.  The  ALV  achieves  autonomy  in  the 
near-term  at  the  expense  of  mobility;  the  ALV  is  currently  capable  of 
traversing  only  obstacle  free  roads  at  relatively  low  speeds.  However, 
just  as  the  Army's  SAT  program  will  evolve  greater  autonomy  over  time,  the 
ALV  will  evolve  greater  maneuverability  over  time.  In  1986  the  ALV  will 
be  capable  of  navigating  roads  with  obstacles,  in  1987  desert  type  of  off 
road  terrain,  and  eventually  more  complex  terrain  at  higher  speeds. 


The  ALV  incorporates  digital  terrain  data  bases,  vision  systems,  and  an 
Inertial  unit  Just  as  the  SAT  does.  In  the  case  of  the  ALV  the  machine 
interprets  the  data  and  automatically  makes  decisions  while  in  the  SAT  a 
combination  of  human  and  machine  processing  is  used.  The  SAT  is  designed 
as  the  Service  program  to  directly  incorporate  the  DARPA  research  results, 
and  the  ALV  will  serve  as  the  primary  source  of  the  technologies  that  will 
enable  the  SAT  to  achieve  greater  autonomy  over  time.  The  SAT  program 
offers  a  continually  evolving  military  capability  in  robotic  vehicles  as 
it  always  retains  full  terrain  mobility,  while  the  ALV  program  seeks 
yearly  breakthroughs  in  the  state-of-the-art  in  autonomous  systems. 

The  principles  of  supervised  autonomy  control  are  currently  under  development 
in  the  Army  Advanced  Ground  Vehicle  Technology  (AGVT)  program,  which  is  a 
precursor  to  the  SAT  program.  Both  FMC  Corporation  and  General  Dynamics, 
under  contract,  are  modifying  their  IR&D  robotic  vehicle  systems  to  incorpo¬ 
rate  today's  ALV  road  following  algorithms.  During  the  summer  of  1986  the 
first  demonstration  of  a  supervised  autonomy  control  of  a  military  vehicle 
will  be  conducted  at  the  ALV  test  site.  The  vehicles  will  perform  a  route 
reconnaissance  mission  scenario  which  will  allow  teleoperation  over  cross 


country  terrain  and  autononoua  road  following  on  paved  road  aur faces  using 
_  the  ALV  algorithms. 

Hapcr  u:'r  • 

bc)o«  th;-  both  of  these  robotic  vehicle  systems  make  use  of  digital 

terrain  data  bases,  and  products  derived  from  these  data  bases,  ia  the 
basis  for  this  paper.  GIS  principles  are  deeply  imbedded  in  the  operation 
of  supervised  or  fully  autonomous  control  systems  for  robotic  vehicles. 
While  GIS  principles  are  much  more  implicit  than  explicit  in  robotic 
vehicles,  their  efficient  and  functional  operation  is  crucial  to  successful 
military  missions.  What  follows  is  a  discussion  of  the  products  and 
procedures  used  in  robotic  vehicle  operations  that  are  dependent  on  GIS 
principles. 


TERRAIN  DATABASES  IN  ROBOTIC  VEHICLE  CONTROL  SYSTEMS 

Terrain  analysis  data  in  digital  form  provide  a  priori  knowledge  of  the 
natural  and  cultural  features  in  an  operational  area  of  a  robotic  vehicle. 
Additional  data  in  digital  form  derived  from  sensors  and  intelligence  data 
supplement  the  terrain  analysis  data.  These  data  provide  a  basis  for 
automating  several  aspects  of  control  systems  for  robotic  vehicles  including 
route  planning  and  graphic  displays. 

Whether  or  not  a  vehicle  is  deployed  in  a  teleoperated  or  supervised  autono¬ 
mous  mode,  its  ability  to  perform  its  mission  depends  on  effective  naviga¬ 
tion  to  achieve  a  task  goal.  The  navigation  process  considers  both  local 
path  planning  functions  and  global  route  planning. 

Automated  route  planning  involves  the  use  of  digital  terrain  analysis  data 
with  knowledge  of  troops  or  vehicles  to  establish  a  route  to  travel  between 
locations.  Route  planning  must  also  consider  multiple  goals,  constraints, 
and  temporal  factors.  Route  planning  can  involve  replanning  in  the  event 
new  knowledge  invalidates  the  route  being  followed. 

ETL  has  developed  a  route  planner  development  workstation  which  serves  as 
a  laboratory  research  tool  to  support  the  development,  understanding,  evalu¬ 
ation,  and  testing  of  automatic  route  planning  algorithms  for  land  based 
robotic  vehicle  applications.  The  workstation  provides  a  structured  and 
efficient  working  environment  for  developing  algorithms.  The  workstation 
technology  has  potential  for  incorporation  in  fielded  applications.  The 
hardware  for  the  route  planner  development  workstation  consists  of  a  Lisp 
machine  with  color  graphics  display  terminal.  The  Lisp  machine  provides  a 
powerful  symbolic  processing  capability  and  is  widely  used  for  artificial 
intelligence  applications. 

The  commander  of  a  robotic  vehicle  must  be  able  to  visualize  the  terrain 
within  his  area  of  responsibility  to  effectively  operate  within  it.  The  basic 
planning  tool  in  use  today  by  the  Army  is  the  large-scale  topographic  map. 
The  commander  must  be  able  to  translate  the  map's  symbology  into  the  reality 
of  the  Earth's  land  form.  This  realization  of  the  form  of  the  terrain  on 
a  rapidly  developing  battlefield  must  be  timely  and  accurate.  Ideally, 
the  map  user  should  be  able  to  visualize  the  terrain's  three-dimensional 


characterlttict  by  aerely  studying  the  asp's  contour  lines.  Unfortunately,  j 
considerable  training,  experience  end  tlae  are  required  to  Interpret 


('j:'.-  >7  1  exactly  what  the  relief  is  like  as  depicted  by  the  asp's  contour  lines.  i 

bcU' A  !  j  j 

The  asp  sheet  Is  not  effective  for  use  as  an  analytical  tool  for  such  opera-  | 
tlons  as  deteralnlng  llne-of-alght  between  terrain  points.  Analytical 
operations  Involving  terrain  data  are  common  In  the  field  as  when  a  comaander  - 
plans  optlaal  positions  for  observation  or  concealaent.  Autoaated  systeas 
Incorporating  digital  terrain  analysis  data  with  appropriate  algorithms 
can  provide  the  commander  of  a  robotic  vehicle  with  three-dimensional 
r  perspective  maps.  The  S-dlmensional  perspective  map,  such  as  shown  in 

I  figure  4,  presents  a  view  of  geographic  information  In  a  fora  which  can  be 

t  comprehended  faster  than  the  traditional  2-dlmen8ional  map.  | 

I  ETL  Is  demonstrating  the  potential  for  use  of  three-dimensional  perspective 

'!  maps  for  use  in  control  systems  for  robotic  vehicles  through  the  develoisnent 

of  a  laboratory  system  consisting  of  a  high-performance  workstation  for 
.  display  of  3-D  graphics  and  associated  software.  The  workstation  accepts 

t  as  input  digital  gridded  elevation  data  which  has  been  converted  to  polygon 

I  form  and  associated  digital  planimetry.  The  digital  data  is  the  basis  for  ' 

r  the  hardware  has  creation  of  a  3-D  perspective  map.  The  sophisticated 

nature  of  the  graphics  hardware  permits  continuous  graphic  manipulation  of 
\\  the  data, 

l' 

Once  loaded  into  the  graphics  workstation,  geographic  data  can  be  displayed 

I  in  3-D  using  perspective  or  oblique  views.  A  special  case  of  the  oblique 

looks  at  the  terrain  surface  from  directly  above  giving  the  appearance  of 
standard  softcopy  2-D  maps.  By  Interactively  changing  the  viewing  parameters 
of  the  graphics  workstation,  the  viewing  position  can  be  changed  to  give  a 
3-d  view. 


I  The  maps  displayed  on  the  3-D  graphics  workstation  provide  the  commander  with 

*  an  additional  Input  with  which  he  can  make  his  own  expert  decisions  with 

.  respect  to  operation  of  the  robotic  vehicle.  The  commander  can  use  the 

'  Interactive  3-D  map  to  generally  view  and  scroll  through  the  geographic  area 

of  operation.  The  perspective  and  oblique  views  of  the  terrain  provide  an 
^  additional  means  for  the  commander  to  select  end  points  for  automated  route 

I  planning  operations  and  look  for  potential  choke  points  and  critical  points, 

r  The  results  of  automated  route  planning  operations  can  be  displayed  on  the 

'  3-D  display  so  that  the  commander  can  decide  as  to  the  effectiveness  of 

the  route.  After  a  route  has  been  selected,  the  commander  can  preview  the 
route  as  it  would  be  seen  from  the  robotic  vehicle  through  surrogate 
;  travel.  While  the  robotic  vehicle  is  actually  traversing  an  area,  the  3-D 

I  display  can  create  graphic  views  which  correspond  with  the  views  being 

transmitted  by  the  video  channel.  Unlike  the  live  video,  the  3-D  views 
can  be  made  to  show  the  predicted  view  ahead  of  the  vehicle. 


[  In  either  type  of  robotic  vehicle  control  schema,  supervised  or  autonomous, 

the  terrain  data  can  be  used  as  a  priori  knowledge  for  interpreting  the  vision 
I  Imagery  for  obstacle  detection  and  avoidance.  Today's  technology  has  not 

solved  the  computer  vision  problem,  which  Is  one  of  the  primary  limitations 
to  providing  a  full  autonomous  vehicle  capability.  The  terrain  data  can 
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be  used  to  assist  the  machine  by  greeting  a  machine  readable  predicted 
_  view  of  the  terrain  that  can  be  correlated  In  some  manner  with  the  actual 

('.ipir  <  "  Image  view  that  Is  to  be  processed.  Knowing  where  the  preplanned  route 

\n'  falls  In  the  image  can  also  be  used  to  narrow  down  the  portion  of  the 

Image  to  be  processed  to  only  that  section  relevant  to  the  vehicle  path. 


Many  of  the  same  functions  that  are  performed  In  supervised  vehicle  control  i 
are  performed  In  autonomous  vehicle  systems  as  well.  The  primary  difference  j 
Is  that  autonomous  systems  perform  their  operations  with  machine  readable 
Information  that  does  not  have  to  be  put  Into  human  readable  form.  Autonomous 
systems  require  efficient  access  to  terrain  data  for  automatically  planning  ; 
a  route  or  replanning  a  route  when  needed.  In  addition  they  also  need  to  j 
query  the  terrain  data  base  for  making  decisions  about  preplanned  routes,  j 

As  described  earlier  the  terrain  displays  can  be  presented  as  a  softcopy  2-D 
map.  Using  this  as  background  Information,  the  preplanned  route  can  be 
overlayed  over  It.  As  the  unmanned  platform  traverses  the  terrain  the 
position  and  attitude  information  provided  by  the  on-board  inertial  unit 
can  be  used  to  generate  a  blinking  cursor  overlay  of  actual  vehicle  position 
and  heading  to  serve  as  a  navigation  aid.  A  history  of  vehicle  movement 
can  easily  be  added  and  the  background  map  must  be  capable  of  being  scrolled 
as  the  cursor  nears  the  edge  of  the  display.  I 


In  autonomous  systems  the  terrain  data  provides  the  descriptive  and  geometric  \ 

Information  to  allow  for  landmark  recognition  and  Inertial  system  updates. 

While  a  human  readable  blinking  cursor  overlay  would  not  be  used,  the  machine 

must  be  capable  of  reasoning  about  the  vehicle's  position  and  movements.  P.. 
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SIMMARY 

This  Introductory  discussion  of  robotic  vehicle  operations  that  either 
explicitly  or  implicitly  Involve  terrain  analysis  data  bases,  and  the 
requisite  geographic  information  systems,  has  hopefully  Illustrated  the 
value  of  these  types  of  data  and  products  to  military  robotics  systems. 

The  challenge  to  the  robotics  research  community  Is  not  Just  how  to  best  plan 
routes  or  generate  the  best  quality  graphics  display,  but  rather  how  to  best 
perform  these  operations  within  the  physical  and  monetary  limitations  of 
robotic  vehicle  environments.  In  an  autonomous  system  the  entire  computa¬ 
tional  capacity  must  reside  on  board  the  vehicle  and  In  a  supervised  system 
the  computational  power  is  divided  between  the  unmanned  platform  and  the 
command  and  control  center.  In  either  case  the  speed,  storage  capacity,  and 
cost  of  equipment  provide  a  constraint  against  which  researchers  are 
trying  to  provide  optimal  geographic  information  systems  capabilities. 

Geographic  Information  systems  must  be  capable  of  scrolling  through  high 
resolution  terrain  data  bases  and  providing  the  Information  In  an  optimal 
data  structure  for  automatic  route  planning  algorithms.  Conventional 
conceptions  of  fixed  size  maps  or  geographical  units,  such  as  one  degree 
squares,  have  to  be  abandoned  in  favor  of  one  contiguous  area  of  operation 
Instantly  available  to  a  machine.  The  CIS  must  be  capable  of  efficient 
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updating  as  new  terrain  knowledge  is  gained  as  the  vehicle  traverses  the ! 

_ terrain.  Tactical  Intelligence  and  envlronnental  data  have  to  be  readily 

k>j;xt  nn<  I  conblned  with  the  terrain  data  to  nost  accurately  portray  the  area  of  i 
bciouihiij  operation.  •  I  I 

line  ' 


Given  that  these  goals  can  he  accomplished  then  geographic  Information  systems 
will  prove  a  most  valuable  tool  In  the  pursuit  of  robotic  vehicles  to  aid 
the  soldier  In  the  field. 


